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LONG-TERM  GOALS 

The  goals  of  this  project  are  to  increase  our  understanding  of  weather  predictability  and  its  advantages 
and  limitations,  and  to  develop  methods  to  provide  more  accurate  forecasts  and  nowcasts  in  complex 
terrain  using  multi-model  ensemble  modeling  techniques  and  special  observations  including  remotely 
sensed  data. 

OBJECTIVES 

The  main  objectives  of  the  study  are:  1)  To  further  develop,  test,  and  continue  operational  forecasts 
using  both  the  real  time  Weather  and  Research  Forecasting  (WRF)  model  (Skamarock  et  al.  2005)  and 
Mesoscale  Model  5  (MM5)  (Grell  et  al.  1994)  with  sub-kilometer  horizontal  resolution  to  support  the 
NOWCAST  system  at  the  Fallon  Naval  Air  Station  (NAS);  2)  To  produce  multi-model  ensemble 
forecasts  using  WRF  and  MM5;  3)  To  subsequently  include  forecasts  from  the  Coastal  Oceanic  and 
Atmospheric  Modeling  Prediction  System  (COAMPS™)  (Hodur  et  al.  1997)  as  part  of  the  multi¬ 
model  ensemble;  4)  To  develop  methods  that  combine  multi-model  ensemble  forecasts  with 
climatological  fields  to  improve  the  skill  of  the  forecasts  and  nowcasts;  and  5)  To  develop  a  framework 
that  complements  the  ensemble  forecasting  to  better  understand  the  sources  of  error  and  uncertainty  in 
dynamical  forecasts  relevant  to  nowcasting  key  parameters  such  as  wind  speed  and  visibility  over  the 
Fallon  NAS  area. 

APPROACH 

This  project  broadly  focuses  on  two  components:  (1)  maintenance  and  data  collection,  quality  control, 
and  analysis  of  data  from  four  special  weather  stations  in  the  Naval  Air  Station  area,  and  (2)  real  time 
mesoscale  operations  and  mesoscale  multi-model  ensemble  forecasts.  The  forecast  system  is  capable 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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of  using  data  from  the  on-site  weather  stations  for  evaluation  and  data  assimilation.  The  results  from 
the  model  runs  are  updated  every  12  hours  and  posted  on  the  dedicated  web  site  with  password 
protection.  The  data  are  also  available  for  the  NAS  use  through  the  web  with  links  to  various 
components  of  the  system  as  shown  in  Figure  1. 
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Figure  1.  An  updated  web  page  of  DRI’s  real  time  MM5  and  WRF  forecasting  systems 

[http://www.  adim.  dri.  edu/J. 


One  of  the  primary  goals  of  this  study  is  to  implement  a  regional/mesoscale  multi-model 
(CO AMPS™,  WRF,  and  MM5)  ensemble  forecasting  system.  One  of  the  ways  to  provide  ensemble 
forecasts  from  a  particular  model  is  to  perform  multiple  model  runs  where  each  run  has  the  same  initial 
and  boundary  conditions,  but  different  physical  parameterization  options.  Although  perturbations  in 
both  the  initial  and  boundary  conditions  should  be  considered  for  operational  forecasting,  studies  such 
as  Stensrud  et  al.  (2000),  Stensrud  (2001),  and  Stensrud  and  Weiss  (2002)  have  shown  that  variations 
based  on  altered  physical  parameterization  alone  lead  to  meaningful  probability  density  functions 
(pdfs)  of  the  forecast  variables.  The  results  are  especially  promising  for  short-term  severe  weather 
forecasts. 

Our  approach  is  based  on  generation  of  an  ensemble  member  set  by  randomly  varying  the  available 
planetary  boundary  layer  schemes  (Eta  PBL,  Burk-Thompson  scheme,  Pleim-Xiu  scheme,  Blackadar 
scheme,  MRF  PBL  scheme,  and  Gayno-Seaman  scheme),  cloud  microphysical  schemes  (Thompson 
scheme,  Eta  microphysics,  Goddard  microphysics,  Lin  scheme,  Reisner  scheme,  and  Dudhia  scheme), 
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radiation  schemes  (Rapid  Radiative  Transfer  model  computations,  Dudhia  scheme,  CCM2  scheme, 
Goddard  scheme,  GFDL  scheme,  and  CAM  scheme  as  used  in  NCAR  CCSM3  climate  model),  and 
cumulus  parameterizations  (Kain-Fristch  scheme,  Betts-Miller  scheme,  and  modified  Grell  ensemble 
scheme)  of  each  model.  A  large  random  set  of  50  WRF  and  50  MM5  ensemble  members  using 
different  physics  options  are  created  by  using  the  built-in  Fortran  random  function  utility.  The  WRF 
and  MM5  control  runs  (non-ensemble  base  states)  use  essentially  similar  physical  parameterization 
schemes. 

The  main  objective  is  to  create  and  analyze  probability  density  functions  (pdfs)  for  variables  from  each 
model  and  then  combine  the  models  (two-model  ensemble)  with  a  total  of  100  ensemble  members. 

The  high-resolution  control  runs  serve  as  a  valuable  means  of  comparison  with  the  ensemble  results 
(Houtekamer  and  Mitchell  1999).  Once  the  ensemble  runs  are  completed,  the  analysis  of  the  ensemble 
set  includes:  (a)  pdf  properties,  statistics,  and  evolution;  (b)  Rank  histograms  (Talagrand  diagrams); 
and  (c)  Evolution  and  spread  of  parameter  trajectories  (“spaghetti  plots”:  Superposition  of  forecast 
isolines  for  the  ensemble  members). 

WORK  COMPLETED 

We  have  made  ensemble  forecasts  with  50  WRF  members  and  50  MM5  members  for  a  period  of 
fifteen  days  (12-27  December  2008).  This  case  was  chosen  because  of  the  dramatic  frontal  passages 
that  occurred  over  NW  Nevada  during  this  period.  Specification  of  initial  and  boundary  conditions 
relied  on  the  GFS  forecasts  (ftp://tgftp.nws.noaa.gov/)  with  a  0.5°  x  0.5°  grid  resolution  were  used  for 
0-180  hours  and  forecasts  from  180-384  hrs  were  used  with  available  2.5  x  2.5°  resolution. 


Figure  2.  The  model  domains  for  the  ensemble  runs. 


The  coarse  domain  covered  most  of  the  eastern  Pacific  and  the  entire  U.S.  with  a  horizontal  resolution 
of  108  km;  the  nested  domain  has  horizontal  resolution  of  36  km.  Both  domains  have  36  vertical 
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levels.  Since  there  is  an  ongoing  discussion  in  the  literature  (see,  e.g.,  Houtekamer  and  Mitchell  1999) 
concerning  the  justification  of  running  ensemble  model  with  many  members  compared  to  a  control  run 
with  high  resolution,  we  have  performed  control  runs  for  WRF  and  MM5  with  an  additional  innermost 
nested  grid  with  12  km  resolution  that  covers  the  entire  states  of  Nevada  and  California.  The  forecasts 
from  the  WRF  and  MM5  ensemble  members  and  control  runs  were  evaluated  using  data  from  a  total  of 
80  upper-air  stations,  where  26  upper-air  stations  were  located  within  the  36  km  resolution  nested 
domain. 


RESULTS 

Figure  3  shows  pdf  ensemble  histograms  for  WRF  and  MM5  at  particular  forecast  times  (2,  5,  10,  and 
15  days). 
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Figure  3.  PDF  histograms  of 500  hPa  forecasted  air  temperature  at  Oakland,  CA.  The  forecasts  at  t 
=  2,  5, 10,  and  15  days  are  shown  in  the  upper-left,  upper-  right,  lower-left,  and  lower-right  corners 
of  the  diagram.  In  addition  to  the  histogram,  the  observation,  the  control  forecast,  and  the  two- 
model  ensemble  mean  are  indicated.  Super-ensemble  mean  is  the 
mean  for  the  two-model  ensemble  forecast. 
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The  WRF  and  MM5  pdfs  exhibit  significant  differences  at  t  =  2d  and  the  combined  ensemble  therefore 
exhibits  a  bi-modal  distribution.  By  t=5d,  the  combined  distribution  does  not  display  a  bi-modal 
structure  and  the  observations  fall  near  the  mean.  Notice  that  the  ensemble  averages  showed  some  skill 
following  radiosonde  observations  even  at  later  forecast  times.  The  correlation  coefficient  between  the 
MM5  [WRF]  ensemble  forecast  mean  and  air  temperature  observations  at  500  hPa  at  Oakland,  CA  is 
0.61  [0.74],  and  between  super-ensemble  mean  and  observations  is  0.70. 

Previous  research  indicates  that  shorter  forecasts  are  dominantly  driven  by  the  dynamics  and  later 
forecasts  driven  by  physical  parameterizations.  Prior  to  the  10-day  forecasts,  the  spread  (amplitude)  of 
each  of  the  models  is  only  about  15  K;  in  the  later  stage  the  spread  increases  to  35-40  K.  To  investigate 
the  dispersive  properties  of  the  ensembles,  Talagrand  diagrams  are  used  as  shown  in  Figure  4.  The 
figure  shows  WRF  and  MM5  rank  histograms  for  the  air  temperature  at  500,  700,  and  925  hPa  at 
Oakland,  CA,  using  radiosonde  observations. 


Ranked  Bins 


Figure  4.  Talagrand  diagrams  for  the  air  temperature  at  500,  700,  and  925  hPa  for  WRF  (dark 
gray)  and  MM5  (light  gray)  based  on  the  26  upper  air  observations  in  the  nested  domain. 


Both  models  somehow  show  larger  under-estimation  than  over-estimation.  Notice  that  each  of  the 
models  was  normalized  by  its  own  frequency  maximum;  however,  WRF  appears  to  have  a  more 
uniform  distribution  than  MM5.  It  is  quite  clear  that  MM5  is  not  able  to  provide  the  same  results  as 
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WRF  at  the  lowest  level.  An  additional  complication  stems  from  the  number  of  available  pairs  at  the 
lowest  level  due  to  high  elevation  complex  terrain.  Gaps  in  the  histograms  at  925  hPa  indicate  that 
even  100  ensemble  members  may  be  insufficient  to  produce  a  uniform  distribution  at  the  lowest  levels. 

Figure  5  shows  temporal  dispersion  in  terms  of  “spaghetti  plots”  of  the  two  ensemble-predicted 
isolines  of  geopotential  heights  5250  and  5500  m  and  isotherms  of  238  and  258  K. 
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Figure  5.  “Spaghetti”  plots  for  the  MM5  and  WRF  ensembles  of  the  5250  m  (blue  lines)  and  5500 
m  (green  lines)  geopotential  heights  and  238  K  (green  lines)  and  258  K  ( blue  lines)  isotherms  at  500 
hPa.  Columns  1  and  3  (Columns  2  and  4)  are  plots  of  geopotential  height  and  temperature, 
respectively  from  MM5  (WRF).  Horizontal  rows  correspond  to  forecast  times  of  2,  5, 10, 

and  15  days. 


Both  model  ensembles  produced  similar  patterns  of  geopotential  heights  up  to  5 -days  and  started  to 
significantly  spread  after  that  time.  The  air  temperature  forecasts  show  much  larger  differences  even  at 
the  2-day  forecast  time.  However,  the  WRF  temperature  ensembles  stay  more  coherent  until  the  end  of 
the  simulation  period.  Better  statistics  for  WRF  indicate  that,  in  this  case,  a  larger  spread  does  not 
necessarily  guarantee  that  the  results  are  better.  However,  from  an  operational  forecasting  aspect,  a 
larger  ensemble  spread  is  generally  more  desirable. 
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Another  way  of  looking  into  the  ensemble  spread  is  to  compare  ensemble  trajectories  for  a  particular 
parameter,  location  and  height.  Figure  6  shows  trajectories  of  individual  ensemble  members  for  the  air 
temperature  and  the  geopotential  height  at  Oakland,  CA,  at  500,  700,  and  925  hPa  with  superimposed 
radiosonde  observations.  It  appears  that  the  spread  characteristics  are  quite  similar  for  both  models  and 
parameters. 


Figure  6.  WRF  (blue  lines)  and  MM5  (green  lines)  ensemble  trajectories  for  the  air  temperature 
(left panels)  and  the  geopotential  height  (right panels)  at  Oakland,  CA,  at  500,  700,  and  925  hPa 
with  observations  (red  dots)  for  the  period  from  12-27  December  2008. 


As  noted  earlier,  the  synoptic  situation  was  characterized  by  frontal  passages.  In  Figure  6,  we  see  that 
the  temperature  drops  at  Oakland  are  evident  between  December  12-17,  2008  and  during  December 
23-25,  2008.  All  ensemble  members  closely  followed  the  frontal  passage  during  the  first  two  days  of 
the  predictions  as  indicated  by  the  observations.  The  dynamics  were  fully  dominant  during  this  period. 
After  the  initial  uniformity,  there  is  a  general  bi-modal  split  in  the  2-5  day  forecast  period  between  the 
WRF  and  MM5  trajectories.  The  ensemble  forecasts  at  the  time  of  second  frontal  passage  (December 
23-25,  2008)  exhibits  large  spread  and  the  spread  does  not  increase  monotonically  with  an  increase  in 
forecast  time.  Notice  that  the  spread  is  quite  similar  at  all  levels  and  it  does  not  appear  to  be  larger  at 
the  lowest  level  where  most  of  the  complexity  in  the  flow  structure  generally  occurs.  Both  models 
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failed  to  reproduce  the  second  cold  advection  at  the  end  of  the  forecast  period,  mainly  at  the  upper 
levels.  This  needs  to  be  investigated  further  in  terms  of  the  propagation  of  the  boundary  conditions 
and  the  general  accuracy  of  the  boundary  conditions  from  the  global  models  for  the  medium  range 
forecasts  of  15  days. 

Bear  in  mind  that  more  models  and  more  ensemble  members  are  expected  to  provide  a  better  forecast; 
an  additional  multi-model  ensemble  set  using  COAMPS™  is  currently  underway.  The  typical  3-day 
control  forecast  obtained  from  models  CO  AMPS™,  MM5  and  WRF  from  December  12  to  December 
15  is  shown  in  Figure  7. 
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Figure  7.  (A)  COAMPS™ ,  (B)  MM5,  and  (C)  WRF  3-day  control  forecasts  of  air  temperature  and 
geopotential  height  at  500  hPa  on  15  December  2008  at  0000  UTC. 


As  part  of  our  effort  to  identify  the  sources  of  systematic  error  in  dynamical  prediction  models,  a 
method  has  been  developed  to  determine  the  sensitivity  of  the  model  output  to  the  elements  of  control 
(initial  and  boundary  conditions,  and  empirical/physical  parameters  of  the  model).  The  process  has 
been  tested  on  a  mixed- layer  model  with  excellent  results  (Lewis  2007;  Laksmivarahan  and  Lewis 
2009).  We  are  exploring  the  feasibility  of  the  procedure  that  determines  the  adjustments  to  the 
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elements  to  minimize  the  forecast  error  using  a  least  square  variational  data  assimilation  strategy  that 
complements  ensemble  forecasting  through  analysis  of  upper-air  and  surface  observations 
complemented  by  satellite  imagery  and  dynamical  simulations  using  WRF.  The  analysis  and 
deterministic  forecast  of  a  short-lived  intense  dust  storm  over  the  Fallon  NAS  is  also  being  conducted 
as  a  preliminary  to  an  ensemble  forecast  that  emphasizes  operational  prediction  of  visibility. 

IMPACT/APPLICATIONS 

Although  ensemble  forecasting  has  been  used  for  global  predictions  at  major  forecasting  centers, 
regional  and  mesoscale  ensemble  forecasting  is  currently  in  research  and  development  stage. 
Furthermore,  high-resolution  multi-model  ensemble  forecasting  holds  promise  for  regional/mesoscale 
models'  structure  through  exploration  of  physical  parameterizations  that  are  necessary  to  improve  high- 
resolution  forecasts  in  complex  terrain.  Although  our  research  is  incomplete,  the  currently  available 
real  time  ensemble  forecasts  are  accessible  by  the  Fallon  Naval  Air  Station  and  will  hopefully  improve 
operational  nowcasts  and  forecasts  crucial  to  the  Navy’s  operations. 

TRANSITIONS 

Both  the  special  set  of  four  weather  stations  in  the  Fallon  area  [  http://www.wrcc.dri.edu  ]  and  the 
ongoing  WRF  and  MM5  operational  forecasting  system  [  http://www.adim.dri.edu  ]  have  been 
developed  as  a  complement  to  the  forecasting  and  nowcasting  at  the  Navy’s  Fallon  Naval  Air  Station. 

RELATED  PROJECTS 

Dr.  Koracin  is  a  co-P.I.  on  ARO  Project  entitled  “Forecasting  of  Desert  Terrain”  where  real-time 
experience  and  expertise  is  facilitating  an  interdisciplinary  project  linking  dust  emission  modeling, 
atmospheric  predictions  and  Lagrangian  Random  Particle  Dispersion  modeling.  Dr.  Koracin  is  a  Lead 
Investigator  for  a  Climate  Modeling  component  of  the  multi-institutional  NSF-EPSCoR  Project  on 
Climate  Change,  where  they  are  developing  new  methods  of  weather  and  climate  forecasting  and  use 
of  satellite  data  assimilation  for  model  evaluation.  They  are  also  investigating  predictability 
limitations  and  chaotic  behavior  in  weather  and  climate  predictions  and  methods  of  downscaling  global 
model  results  to  regional,  mesoscale,  and  microscale  applications.  As  a  Principal  Investigator  on  a 
DOE-NREL  Wind  Energy  project,  he  is  improving  high-resolution  forecasts  in  complex  terrain.  Dr. 
Koracin  is  a  Principal  Investigator  on  a  newly  awarded  DOE-Office  of  Science  project,  Simulating 
Climate  on  Regional  Scale:  North  Pacific  Mesoscale  Coupled  Air-Ocean  Simulations  Compared  with 
Observations.  The  main  task  is  to  fully  couple  the  ocean  model  (POP)  and  the  atmospheric  model 
(WRF)  over  the  open  ocean  and  coastal  regions. 

Dr.  Lewis  is  involved  in  two  projects  that  complement  this  ensemble  research;  (1)  variational  analysis 
used  to  identify  sources  of  error  in  dynamical  prediction,  and  (2)  analysis  and  prediction  of  dust  storms 
over  western  U.S.  Both  projects  are  supported  by  NOAA  and  this  ONR  project. 
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